ED 057 080 
AUTHOP. 
TITLE 


PUB DATE 
NOTE 


EDRS PRICE 
DESCRIPTORS 


IDENTIFIERS 


ABSTRACT 


DOCUMENT RESUME 
TM 000 923 


Seeman, Alice 

Feedback and the Measurement of Change. 

Feb 71 

5p.3 From symposium "Feeding Back Information 
Collected From School Staffs," American Educational 
Research Association, New York, New York, February 
1971 


MF-$0.65 HC-$3.29 
*Educational Change; Educational Improvement; 
Fducational Problems; *Feedhack; *Formative 
Evaluation; *IYnformation Utilization; Research 
Methodology; *Research Problems; Research 
Utilization; Symposia 
*League of Cooperating Schools 

H 


Feedback is discussed in relation to measuring the 


change process, a three-step procedure of dialogue, decision-making, 
and action-taking. Measurement problems resulting from feedback are 


presented. 
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FEEDPACK AND THE MEASUREMENT OF CHANGE 


[t is customary to examine change from a substantive perspective, 
looking (for cxample) at the effects of forced desegregation, the cffectiveness 
of team teaching, or the consequences of various curriculum or administrative 
alterations. It seems cqually reasonable, however, to focus attention not on 
any particular change but on the process of change itself, That is the perspec- 
tive in which the research on the League of Cooperating Schools is taking place; 
and I would like to suggest how the use of feedback in that contexc (with its 
emphasis on the League and on the process of change) creates some very 
special problems for the researcher. 

Our assumpcion was that schools would ve abte vw respoud mose erfec- 
tively if they had incorporated a process of change into their institutional 
system. That process was conceived to be a three-step procedure ef dialogue, 
decision-making, and action-taking (DDA)-~-each of, these being, in turn, inecas- 
urable along various dimensions (e.g. their flexibility, consiatency, the degree 
of consensus involved, ete.). The details regarding this measurement scheme 
are net vital here. The point is that given this interest in process, what feed- 
back as a technique of social intervention does (especialiy in a League context) 
is to intensify many of the standard methodological problems in research. How 
so? 

(1) To the extent that feedback emphasizes the importance of the DDA 
process, and implicitly praises or criticizes the level of DDA in a given school, 


ir hecemes difficult ji subsequent measurements to discriminate “test-wise" 
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responses from “real-change". Knowlcdse of results in itself affects later 
measurements. But beyond that, enlisting the subject in the research enter- 
prise involves teaching him what the "correct" response is--in this case, 
teaching him something of the presumed significance of the DD.\ process and 
developing a "DDA orientation”. The standard methodological preblem in most 
research is to keep the respondent ignorant of what is at stake, even to deccive 
him where necessary for the sake of the experinent. Our feedback intensifies 
that problem by a "show-and-tell" approach which may he ethically more 
elegant but makes researchable propositions infinitely more difficult to demon- 
strate. 

(2) Acommon answer to the problem of subject awareness is to develop 
multiple and cquivalert mcasures of the variable beuiy reviewed. Thus, where 
feedback teaches the right verbal response, one would search for alternative 


non-verbal indices--what Webb, et al” have called “uucbtrusive” (essentially non- 


reactive) measures. However, two problems emerge: 

(a) Since the emphasis is upon the change process, it peenines 
excecdingly difficult to find unobtrusive measures--more difficult, in any 
event, than it might be if the criterion variable were a substantive rather than 
a processual cne. Measures of process are notoriously elusive: the difference 
between measuring an achievement score and measuring the teaching process 
that produced it niccly illustrates the problem. We have tried to get data from 
various sources-“from the teachers’ descriptions (via questionnaires) of the 


process that was involved ima particular school change; from the documents of 
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desienated reparters (at cach school) dereiting, month-by-month, the events 
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relevant to DDA in their schools; from interviews with teachers in each school; 


and from observations at staff meetings in cach school. But it remains true 
that unobtrusive measures of dialogue, decision, and action are hard to come 
by. 

(b) To complicate the matter further, our feedback stratezy aims 
at thoroughgoing change, involving commitment to a very basic process in the 
schools. If the understanding we convey is thoroughgoing, the researcher will 
have no better comprehension of this change process than the teachiag staff 
docs. To the degree that this is so (and in a sense the aim of feedback is to help 
make it so) there are no unobtrusive measures--i.c. staff members can create 
and recognize such measures as well as the researcher can. [f thac situation 
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lostrable rostizotion of odventional 
goals but a cousidcrable problem for the research objectives. 

(3) Customary research practice secks to insulate one ics. group from 
another (e.g. in the laboratory, warning against discussing the experiment with 
others). In the League, external communication is built into the conception of 
the change process. The informat‘on fed back is supposed to be communicated 
beyond the boundaries of the given school. The League serves both as a channel | 
of communication of ideas and as a source of mutual support for efforts at 
innovation, Whatever the desirable educational goals that may be served by 
this arrangemenc, it raises problems for the research in assessing what effect 
any given feedback experience has had upon the school. 

(4) Feedback atso introduces a calculaved and probably powertul source 


of unreliahilin. Tr does so by providiag the teachers with now standards of 
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reference concerning what is high, low, or average standing on various meas- 
ures of dialogue, decision, and action. Re-measurement after fecdback is likely 
to show the effects of this new awareness: e.g, what was once thought to be 
"very much" dialogue may be rated as only "average". In effect, the measuring 
device is "rubberized" by the implicit shifting of standards for judgment. 

To conclude, I wish I could offer solutions to these problems, but such 
solutions are not so readily found. The general approach is nicely described in 
Kidder and Campbell's suggestion that: 

"Rather than speak of ‘subjects’ who must be coerced or cajoled 

into cooperating and whose defenses must be overcome, we might 


eturn to the view of the German psychologists who treated their 
respondents as Versuchspersonen, or fellows in research" 


erhans the basic sclution is alsoa very gZonerai one: Tue full awareness lat 
bringing the research subject into the investigative process, through teedback 
or other procedures, presents both opportunity and a formidable research 
challenge. 
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